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Claims 



[d] A method for modeling and simulating software running 
interactively directly or indirectly on at least one digital 
computer, comprising the steps of: 
providing a display; 

providing an input, wherein said input connects at least 
one of a pointing device, a keyboard and external inter- 
active devices to said software; 
providing an output, wherein said output connects a 
block of memory to said display; 
providing a software controller, wherein said software 
controller is a programmable agent controlling said soft- 
ware to perform tasks; 

providing a software modeling process, wherein said 
software modeling process models an interaction pro- 
cess between said software and said software controller, 
further comprising the sub-steps of: 

(a) connecting said software controller with said soft- 
ware through said input and said output of said soft- 
ware, 

(b) controlling said software by said software con- 
troller, and 

(c) identifying a model of said software on-line; and 



providing a software simulation process, wherein said 
software simulation process simulates said interaction 
process between said software and said software con- 
troller, further comprising the sub-steps of: 

(d) connecting said software controller with said 
model of said software through a simulated input 
and a simulated output of said model of said soft- 
ware, 

(e) controlling said model of said software by said 
software controller, and 

(f) simulating said interaction process between said 
software and said software controller without said 
software presence; and 

wherein said software simulation process is a new soft- 
ware that comprises said model of said software and 
said software controller. 

[c2] The method of claim 1, further comprising the step of: 
providing a discrete sampling domain, wherein said dis- 
crete sampling domain is a finite integer sequence K with 
a current sampling k indicating the most recent sam- 
pling. 

[c3] The method of claim 2, further comprising the steps of: 
providing a software dynamic system to represent said 
software modeling process, wherein said software dy- 
namic system is a discrete system defined over said dis- 



crete sampling domain K; 

wherein said software simulation process simulates said 
software dynamic system. 

[c4] The method of claim 3, further comprising the steps of: 
factoring said software into an algorithm engine and a 
cursor engine; 

identifying a model of said algorithm engine and a 
model of said cursor engine in said software modeling 
process; 

controlling said model of said algorithm engine and said 
model of said cursor engine in parallel by said software 
controller in said software simulation process; and 
combining an output of said model of said algorithm en- 
gine and an output of said model of said cursor engine 
as said simulated output of said model of said software 
in said software simulation process. 

[c5] The method of claim 4, wherein said cursor engine is 
modeled, identified and simulated, wherein the method 
further comprises the steps of: 

providing a cursor function, wherein said cursor function 
calculates a cursor output based on a cursor position, a 
data structure and a plurality of cursor images; 
wherein said cursor function models said cursor engine 
analytically; 

providing a cursor engine modeler, wherein said cursor 



engine modeler identifies said data structure and said 
plurality of cursor images while said cursor engine is 
controlled by said software controller in said software 
modeling process; 

wherein said model of said cursor engine under control 
of said software controller comprises said cursor func- 
tion with said identified data structure and said identi- 
fied cursor images; 

wherein said model of said cursor engine simulates said 
cursor engine in said software simulation process, fur- 
ther comprising the sub-steps: 

controlling said model of said cursor engine by said 
software controller in said software simulation pro- 
cess; 

calculating said cursor output by said cursor function 
in said software simulation process based on, 
said identified data structure, 
said identified cursor images, and 
said cursor position that is supplied by said soft- 
ware controller; 
whereby said cursor engine is simulated by said model 
of said cursor engine that is controlled by said software 
controller in said software simulation process without 
said software presence. 

The method of claim 4, wherein said algorithm engine is 



modeled and identified, wherein the method further 
comprises the steps of: 

providing an initial state of said algorithm engine, 
wherein said initial state of said algorithm engine is 
sampled at the first output of said software; 
providing a update function, wherein said update func- 
tion is a sequence of discrete updates over said discrete 
sampling domain; 

wherein said update function with said initial state model 
said algorithm engine; 

providing an algorithm engine modeler, wherein said al- 
gorithm engine modeler samples said sequence of dis- 
crete updates while said software is controlled by said 
software controller in said software modeling process; 
wherein said model of said algorithm engine identified 
comprises said sampled initial state and said sampled 
sequence of discrete updates as a modeled algorithm 
engine. 

The method of claim 6, wherein said algorithm engine 
modeler is controlled by said software controller and 
said algorithm engine. 

The method of claim 7, wherein said algorithm engine 
modeler is controlled by two sub-samplers, wherein the 
method further comprises the steps of: 
providing a direct input and output sampler; 



providing an internal computation sampler; and 
wherein said direct input and output sampler and said 
internal computation sampler run mutually exclusively. 

[c9] The method of claim 8, wherein said direct input and 
output sampler controls said algorithm engine modeler 
to sample at least one discrete update induced directly 
by a command from said software controller, wherein the 
method further comprises the steps of: 
issuing said command by said software controller; 
waiting for T(a), wherein said a < 1; 
calling said direct input and output sampler by said soft- 
ware controller; 

triggering said algorithm engine modeler by said direct 
input and output sampler to sample said discrete update 
D(k+oc)-D(k); 

holding said command by said software controller; 
advancing said current sampling k by one; 
modeling D(k+a) as D(k+1). 

[do] The method of claim 9, wherein said direct input and 
output sampler is callable by said software controller 
programmatically. 

[cH] The method of claim 8, wherein said internal computa- 
tion sampler controls said algorithm engine modeler to 
sample at least one discrete update, wherein the method 



further comprises the steps of: 

(a) providing an internal computation flow, wherein said 
internal computation flow is executed by said algorithm 
engine; 

(b) providing at least one anchored point in said internal 
computation flow, wherein said anchored point is an in- 
ternal computation unit of said algorithm engine with its 
computation state transition to be sampled precisely; 

(c) providing a software sensor, wherein said software 
sensor is attached into said anchor point; 

(d) setting up said software sensor with a numberOfS- 
tates argument, wherein said numberOfStates argument 
specifies computation state transitions to be observed; 

(e) driving said algorithm engine to said internal compu- 
tation unit; 

(f) calling said software sensor before said internal com- 
putation unit is processed; 

(g) processing said internal computation unit; 

(h) calling said software sensor after said internal com- 
putation unit is processed. 

(i) subtracting said numberOfStates argument by one; 
(j) if said numberOfStates argument is not zero, going 
back to (e); otherwise 

(k) removing said software sensor. 

[d2] The method of claim 11, wherein said software sensor is 



an Event Probe that may perform any programmed func- 
tion including triggering said sampling by said algorithm 
engine modeler, wherein the method further comprises 
the steps of: 

providing an anchored event, wherein said anchored 
event is a window message; 

wherein said anchored event processing is said anchored 
point; 

modifying a callback function address of an internal win- 
dow message processing; 

driving said algorithm engine to said internal window 
message processing; 

providing a dispatch unit, wherein said dispatch unit 
checks a current message with said anchored event, if a 
match is found, then, 

calling a BeforeEvent Probe, wherein said BeforeEvent 
Probe is said Event Probe, and 
processing said anchored event processing, 
calling said AfterEvent Probe, wherein said AfterEvent 
Probe is said Event Probe, and 
returning from said internal window message pro- 
cessing; 

otherwise, processing an internal window message pro- 
cess. 

[d3] The method of claim 11, wherein said software sensor is 



an API Probe that may perform any programmed function 
including triggering said sampling by said algorithm en- 
gine modeler, wherein the method further comprises the 
steps of: 

(a) providing a marker function, wherein said marker 
function is an application programming call that is mod- 
eled as said anchored point within said internal compu- 
tation flow; 

(b) modifying a binary execution image of said software 
to reroute a calling to said marker to a new function, 
wherein said new function includes at least one of said 
API probes and a call to said marker function; 

(c) driving said algorithm engine to said internal compu- 
tation flow; 

(d) proceeding to said marker function; 

(e) calling a BeforeMarker probe before said marker 
function is processed, wherein said BeforeMarker is said 
API Probe; 

(f) processing said marker function; 

(g) calling an AfterMarker probe after said marker func- 
tion is processed, wherein said AfterMarker is said API 
Probe; 

(h) subtracting said numberOf States argument by one; 

(i) if said numberOf States argument is not zero, going 
back to (c); otherwise 

(k) controlling said algorithm engine to stop said pro- 



cessing of said internal computation flow; 

(I) removing said BeforeMarker and AfterMarker probes. 

[d4] The method of claim 11, wherein said step of providing 
at least one anchored point in said internal computation 
flow is a part of an analytical modeling of said algorithm 
engine. 

[d5] The method of claim 11, further comprising the step of: 
providing a controlled target, wherein said controlled 
target is said software in said software modeling pro- 
cess, or said model of said software in said software 
simulation process; 

providing an input of said controlled target, wherein said 
input of said controlled target is said input of said soft- 
ware in said software modeling process, or said simu- 
lated input of said model of said software in said soft- 
ware simulation process; 

providing an output of said controlled target, wherein 
said output of said controlled target is said output of 
said software in said software modeling process, or said 
simulated output of said model of said software in said 
software simulation process. 

[d6] The method of claim 15, wherein said software controller 
further comprises, 

a work-flow controller, wherein said work-flow con- 



troller controls said controlled target to manipulate at 
least one user interface; and 

a work-space controller, wherein said work-space con- 
troller controls said controlled target to perform at least 
one design function. 

[d7] The method of claim 16, wherein said work-space con- 
troller comprises at least one of a coordinate controller 
and a functional controller. 

[d8] The method of claim 17, wherein said coordinate con- 
troller performs said design function, wherein the 
method further comprises the step of: 
providing a list of coordinates, wherein said list of coor- 
dinates are used to drive said input of said controlled 
target. 

[d9] The method of claim 17, wherein said functional con- 
troller performs said design function, wherein the 
method further comprises the steps of: 
providing at least one modelable function, wherein said 
modelable function is called with parameters to calculate 
coordinates; 

wherein said coordinates are used to drive said input of 
said controlled target. 

[c20] The method of claim 19, further comprising the step of: 



providing at least one software actuator, wherein said 
software actuator connects said software controller to 
said controlled target. 

[c21] The method of claim 20, wherein said software actuator 
models at least one graphical user interface element of 
said controlled target. 

[c22] The method of claim 21, wherein said software actuator 
comprises a work-flow actuator interfacing to said work- 
flow controller of said software controller, wherein the 
method further comprises the steps of: 
sending a high level command from said work-flow con- 
troller to said work-flow actuator, wherein said high 
level command comprises a tag and an input command; 
calculating a valid region based on said tag by said 
work-flow actuator; 

translating said command to at least one primitive action 

by said work-flow actuator; 

if said command is for said pointing device, then 

computing a target position by said work-flow actu- 
ator based on said valid region, 
driving said input of said controlled target to said 
target position by said work-flow actuator, and 
applying said primitive action to said target by said 
work-flow actuator; 

else, 



applying said primitive action to said valid region by 
said work-flow actuator. 

[c23] The method of claim 22, wherein said step of calculating 
said valid region based on said tag further comprises the 
steps of: 

if running in said software modeling process, then 
powering said valid region by said software, 
identifying said valid region based on said tag on- 
line, 

calculating said valid region, and 
saving said valid region with said tag as a part of said 
model of said software; 
otherwise, running in said software simulation process, 
simulating said valid region by said model of said 
software, 

retrieving said valid region based on said tag from 
said model of said software, and 
calculating said valid region. 

[c24] The method of claim 23, wherein said step of identifying 
said valid region based on said tag in said software 
modeling process further comprises the steps of: 
providing a unique textual name, wherein said unique 
textual name is said tag; 

providing a unique relation, wherein said unique relation 
is coded in a string and assigned as said unique textual 



name; 

finding a unique handle from said software based on 
said tag, wherein said handle is uniquely located by said 
unique relation that is coded in said tag; 
identifying said valid region based on said unique handle 
that is powered by said software. 

[c25] The method of claim 24, wherein said unique relation is 
a parent-children-sibling relation that is defined 
uniquely by said graphical user interface element of said 
software. 

[c26] The method of claim 21, wherein said software actuator 
comprises a work-space actuator interfacing to said 
work-space controller of said software controller, 
wherein the method further comprises the steps of: 

(a) sending a high level command from said work-space 
controller to said work-space actuator; 

(b) calculating a coordinate by said work-space actuator; 

(c) translating said high level command to at least one 
primitive action by said work-space actuator; 

(d) driving said input of said controlled target to said co- 
ordinate by said work-space actuator; 

(e) applying said primitive action to said coordinate by 
said work-space actuator; 

(f) going back to (b) if at least one coordinate remains. 



[c27] The method of claim 26, wherein said step of calculating 
said coordinates further comprises the steps of: 
providing a list of coordinates, wherein said list of coor- 
dinates is an array of elements with each element having 
a coordinate (x,y); 

wherein said coordinate is retrieved sequentially from 
said array until the end of said array is reached. 

[c28] The method of claim 26, wherein said step of calculating 
said coordinate further comprises the steps of: 
providing at least one modelable function, wherein said 
modelable function calculates said coordinate based on 
at least one parameter that is supplied by said software 
controller; 

wherein said modelable function implemented by said 
work-space actuator is a mathematical function that is 
scalable. 

[c29] The method of claim 21, wherein said software actuator 
is a system actuator if a system standard graphical user 
interface element is modeled by said software actuator. 

[c30] The method of claim 21, wherein said software actuator 
is a custom actuator if a one custom-built graphical user 
interface element is modeled by said software actuator. 

[c31] The method of claim 29, wherein said system actuator is 



reusable universally for a plurality of said software mod- 
eling processes and said software simulation processes. 

[c32] The method of claim 30, wherein said custom actuator is 
reusable for a plurality of said software controllers for 
said software modeling process and said software simu- 
lation process. 

[c33] The method of claim 23, wherein said software actuator 
installs at least one software sensor. 

[c34] The method of claim 23, further comprising the steps of: 
providing a running context of said controlled target, 
wherein said running context is dynamically recon- 
structed by said software actuator in said software simu- 
lation process; 

providing a structured input and output, wherein said 
structured input and output structuralizes said input and 
output of said controlled target based on said running 
context; 

providing a feedback from said controlled target to said 
software controller through said structured output of 
said controlled target; 

inferring a causal relationship from said running context. 

[c35] The method of claim 20, further comprising the steps of: 
modularizing said software modeling process and said 



software simulation process; 

replacing said software with said model of said software 
that is identified in said software modeling process; 
switching from said software modeling process to said 
software simulation process; 

preserving said connection between said software con- 
troller and said software actuator so that said software 
controller is kept invariant in said modeling process and 
said simulation process. 

[c36] The method of claim 20, wherein said algorithm engine 
is simulated, wherein the method further comprises the 
steps of: 

controlling said modeled algorithm engine by said soft- 
ware controller; 

simulating said algorithm engine. 

[c37] The method of claim 36, wherein said step of controlling 
said modeled algorithm engine in said software simula- 
tion process further comprises the steps of: 
providing a Dynamic Update, wherein said Dynamic Up- 
date simulates said output of said algorithm engine; 
reconstructing an output of said modeled algorithm en- 
gine, wherein said output of said modeled algorithm en- 
gine is updated by said Dynamic Update in sync with said 
software controller; 

providing at least one simulated sampler, wherein said 



simulated sampler simulates said sampler used in said 
software modeling process; 

controlling said Dynamic Update to update one output of 
said modeled algorithm engine by said simulated sam- 
pler if said sampler samples one discrete update of said 
algorithm engine in said software modeling process. 

[c38] The method of claim 37, wherein said simulated sampler 
is controlled by said software controller through said 
software actuator. 

[c39] The method of claim 38, wherein said simulated sampler 
is a simulated direct input and output sampler that sym- 
metrically simulates said direct input and output sampler 
used in said software modeling process. 

[c40] The method of claim 38, wherein said simulated sampler 
is a simulated internal computation sampler that sym- 
metrically simulates said internal computation sampler 
used in said software modeling process. 

[c41] The method of claim 37, wherein said Dynamic Update 
downloads and streams from an external source to up- 
date said output of said modeled algorithm engine. 

[c42] The method of claim 41, wherein said external source is 
at least one local file. 



[c43] The method of claim 41, wherein said external source is 
at least one Internet server. 

[c44] The method of claim 39, wherein said software actuator 
drives said simulated direct input and output sampler, 
wherein the method further comprises the steps of: 
modeling a sampling that is induced directly by said 
command from said software controller; 
simulating said sampling by calling said simulated direct 
input and output sampler. 

[c45] The method of claim 39, wherein said software actuator 
simulates said internal computation flow of said algo- 
rithm engine, wherein the method further comprises the 
steps of: 

modeling a sampling event, wherein said sampling event 
samples after or before said internal computation unit is 
processed; 

simulating said sampling event by calling said simulated 
internal computation sampler. 

[c46] The method of claim 38, further comprising the step of: 
creating a simulated discrete sampling domain from the 
execution flow of said software controller in said soft- 
ware simulation process, wherein said simulated discrete 
sampling domain is identical causally to said discrete 
sampling domain. 



[c47] The method of claim 46, further comprising the steps of: 
providing a compressing transformation on said simu- 
lated discrete sampling domain, wherein said compress- 
ing transformation reduces a time duration between two 
samplings in said software simulation process while 
causality is preserved; 

providing a stretching transformation on said simulated 
discrete sampling domain, wherein said stretching trans- 
formation extends a time duration between two sam- 
plings in said software simulation process while causality 
is preserved. 

[c48] The method of claim 3, wherein said software modeling 
process is a software modeling automation that runs au- 
tonomously. 

[c49] The method of claim 48, wherein said software simula- 
tion process is a software simulation automation that 
runs autonomously, wherein the method further com- 
prises the step: 

providing an output of said software simulation automa- 
tion, wherein said output of said software simulation au- 
tomation is said output of said model of said software 
that is manipulate. 

[c50] The method of claim 49, wherein said software Simula- 



tion automation is augmented with additional computa- 
tion while said software dynamic system is preserved. 

[c51] The method of claim 50, wherein the step of augmenting 
said software simulation automation further comprises 
the steps of: 

providing an interaction input component H, wherein 
said interaction input component H engages a user to 
interact with said software simulation automation; 
providing an index component G, wherein said index 
component G controls visibility of said output of said 
software simulation automation; 
providing a programmable extension component E, 
wherein said programmable extension component E ex- 
tends programmatically said software simulation au- 
tomation with additional computational process. 

[c52] The method of claim 51, wherein said interaction input 
component H is inserted between said output of said 
software controller and said input of said model of said 
software, wherein the method further comprises the 
steps of: 

(a) updating said software simulation automation at a 
current sampling k; 

(b) executing a programmed interaction by said software 
controller; 

(c) updating a running context, wherein said running 



context includes a valid region, a pointing device status, 
a keyboard status, and a current command which are 
generated from said programmed interaction by said 
software controller; 

(d) transferring execution of said software simulation 
automation to said H; 

(e) waiting for an interaction input from said user; 

(f) receiving said interaction input from said user; 

(g) comparing said interaction input with said running 
context; 

(h) if said interaction input is matched with said running 
context, then continuing said execution of said software 
simulation automation; 

(i) otherwise, going back to (e). 

[c53] The method of claim 51, wherein said interaction input 
component H is independent of said software simulation 
automation and said H is applicable universally to any 
software simulation automation. 

[c54] The method of claim 51, wherein said interaction input 
component H can be plugged or unplugged into said 
software simulation automation programmatically so that 
said software simulation automation can run interactively 
or automatically. 

[c55] The method of claim 51, wherein said index component 



G indexes said software simulation automation selec- 
tively, wherein the method further comprises the steps 
of: 

(a) providing at least one index set K , wherein said K is 
a subset of said discrete sampling domain K; 

(b) updating said output of said software simulation au- 
tomation in a memory that is invisible; 

(c) providing a gated output-mapping function, wherein 

said gated output-mapping function controls copying 

from said memory to said display based on said index 

set K; 
i 

(d) running said software simulation automation; 

(e) updating a current output of said software simulation 
automation at a current sampling k in said memory; 

(f) if said current sampling k e K , then, 

copying said current output from said memory to 
said display by said gated output-mapping function, 
continuing said simulation; 

(g) otherwise, 

advancing said current sampling k by one, 

going back to (e) as fast as computationally possible. 

[c56] The method of claim 55, further comprising the steps of: 
providing first textual term, wherein said first textual 
term is associated with a sampling k ; 
providing second textual term, wherein said second tex- 



tual term is associated with a sampling k^, 
providing at least one pair of descriptive textual terms 
for searching, wherein one in said descriptive textual 
terms is said first textual term, and said other in said 
descriptive textual terms is said second textual term; 
conducting searches based on said descriptive textual 
terms to find two samplings k and k^, 
running said software simulation automation; 
updating said current output of said software simulation 
automation in said memory; 
if said current sampling k >= kl and k < k2, then 
copying said output of said software simulation au- 
tomation from said memory to said display by said 
gated output-mapping function; 
selectively running at least one segment of simulation 
from k to k 2 -l visually while computationally running 
said software simulation automation internally. 

[c57] The method of claim 51, wherein said programmable ex- 
tension component E extends additional computation 
into said output of said software simulation automation, 
wherein the method further comprises the steps of: 
providing a new programmable function, wherein said 
new programmable function implements a sub-process; 
updating a current output of said software simulation 
automation at said current sampling k; 



extending said new sub-process with said current output 
as an initial state and a next output of said software 
simulation automation as a final state; 
computing at least one sub-state with said new pro- 
grammable function, wherein said sub-state in said sub- 
process updates over said output of said software simu- 
lation automation; 

updating said next output of said software simulation 
automation at said current sampling k+1. 

[c58] The method of claim 57, wherein said new pro- 
grammable function is a Microsoft Internet Explorer 
Browser component, wherein the method further com- 
prises the steps of: 

providing a DHTML page, wherein said DHTML page is 
loaded by said Microsoft Internet Explorer Browser com- 
ponent; 

providing a Document Object Model (DOM), wherein said 
Document Object Model is programmed in said DHTML 
page; 

providing a URL link, wherein said URL link is encoded in 
said DHTML page; 

providing a target region, wherein said target region is 
defined by said Document Object Model; 
clicking or typing in said target region; 
navigating to said URL link. 



[c59] The method of claim 52, further comprising the steps of: 
providing a controlled target, wherein said controlled 
target is said Microsoft Internet Explorer Browser com- 
ponent; 

lively running said Internet Explorer Browser component 
in said software simulation automation; 
programming web browsing activities by said software 
controller; 

engaging said user with said web browsing activities in- 
teractively or automatically. 

[c60] The method of claim 51, further comprising the steps of: 
providing a software amplification device, wherein said 
software amplification device comprises 

said software controller, 

said interaction input component H, 

said index component G, and 

said programmable extension component E; 
wherein said software amplification device connects with 
said model of said software in a top loop; 
feeding an augmented output of said model of said soft- 
ware through said software amplification device; 
sensing said augmented output by a user as an input 
from said software amplification device; 
engaging said user with said software amplification de- 
vice in a bottom loop; 



making an interaction movement by said user as an out- 
put to said software amplification device; and 
interacting said model of said software by said user 
through said software amplification device. 

[c61] The method of claim 60, further comprising the steps of: 

(a) running a sub-process in said programmable exten- 
sion component E to prepare said user to take a right in- 
teraction, wherein said sub-process implements at least 
one programmable function; 

(b) executing a programmed interaction, wherein said 
programmed interaction is performed by said software 
controller; 

(c) updating a running context, wherein said running 
context includes a valid region, a pointing device status, 
a keyboard status, and a current command which are 
generated from said programmed interaction by said 
software controller; 

(d) waiting in said interaction input component H until 
said user taking a right interaction input against said 
running context; 

(e) scoring said interaction input by said H; 

(f) if said score is higher than a level assigned to a next 
output of said software simulation automation, then un- 
der control of said index component G, 

updating said next output of said software Simula- 



tion automation invisibly, 

forgoing a next sub-process, 

canceling a next programmed interaction; 

(g) otherwise, visually updating said next output of said 
software simulation automation; 

(h) going back to step (a); 

(i) or, reaching a final of said software simulation au- 
tomation. 

[c62] The method of claim 60, wherein said interaction input 
component H, said index component G, and said pro- 
grammable extension component E are implemented in 
modular ways so that each component can be bypassed 
programmatically. 

[c63] The method of claim 51, further comprising the steps of: 
providing a software intelligence device, wherein said 
software intelligence device comprises, 

said model of said software, 

said interaction input component H, 

said index component G, and 

said programmable extension component E; 
wherein said software intelligence device connects with 
said software controller; 

wherein said model of said software further comprises 
a setup part, 

a modeled algorithm engine part, and 



an output part; 
wherein said setup part is simulated invisibly under con- 
trol of said index component C; 

wherein said modeled algorithm engine part is extended 
with an input reasoning by said programmable extension 
component E; 

wherein said output part is extended with an output rea- 
soning by said programmable extension component E; 
interactively simulating modeled algorithm engine 
through said interaction input component H by a human 
user. 

[c64] The method of claim 51, further comprising the steps of: 
providing a software-2, wherein said software-2 com- 
prises an automation loop and an interaction loop; 
wherein said automation loop simulates and augments a 
modeled software dynamic system that runs au- 
tonomously; 

wherein said interaction loop engages a human user in- 
teractively into said automation loop; 
wherein said interaction loop couples with said automa- 
tion loop programmatically. 

[c65] The method of claim 64, further comprising the steps of: 
providing at least one web server; 
providing a plurality of same software-2, wherein said 
plurality of same software-2 are connected to said web 



server through the Internet; 

wherein each of said plurality of same software-2 is run 
interactively by a user locally; 

providing a Master machine, wherein said Master ma- 
chine is selected from said plurality of software-2; 
providing a Master, wherein said Master runs said Master 
Machine; 

providing a group of Learner machines, wherein said 
group of Learner machines are the rest of said plurality 
of software-2; 

providing a group of Learner, wherein each Learner in- 
side said group of Learners runs each Learner machine 
inside said group of Learner machines; 
providing a HTTP request/response protocol over the In- 
ternet, wherein said HTTP request/response protocol is 
used to control the execution of said group of Learner 
machines by said Master machine; 
synchronizing said state of said Learner machines with 
said state of said Master machine interactively by said 
Master through said web server; 

communicating among said plurality of software-2 ma- 
chines interactively through the Internet in real-time. 



[c66] The method of claim 65, further comprising the steps, 
communicating between said Master machine and said 
web server, further comprising the sub-steps of: 



(a) said Master machine transiting to a state, wherein 
said state expects an interaction input from said 
Master, 

(b) said Master machine posting a request including a 
current sampling k to said web server, 

m 

(c) said Master machine waiting for a response from 
said web server 

(d) said web server recording said request informa- 
tion including said k , 

m 

(e) said web server sending said response to said 
Master, and 

(f) releasing said Master machine to continue said 
simulation, 

(g) said Master machine pausing said simulation for a 
local interaction input from said Master in said local 
interaction input component H, 

(h) said Master machine exchanging message with 
said group of Learner instantly through the Internet 
in real-time while said local interaction input compo- 
nent H waits for said interaction input, 

(i) said Master making said interaction input to said 
simulation, and 

(j) said Master machine continuing said simulation; 
and 

communicating between any one of said Learner ma- 
chines and said web server, further comprising the sub- 



steps of: 

(k) said Learner machine transiting to a state, 
wherein said state expects an interaction input from 
said Learner, 

(I) said Learner machine posting a request including a 
current sampling k to said web server, 
(m) said Learner machine waiting for a response from 
said web server, 

(n) said web server comparing between said k and 

said k of said Master machine recorded most re- 
in 

cently, 

if k < k then, 

m 

web server sending said response to said Learner 
machine by immediately, 

said Learner machine continuing said simulation, 
and 

if k >= k , then, 

m 

web server suspending said response to said 
Learner machine, and 

said Learner machine blocking said simulation, 
if k == k , then, 

m 

said Learner machine exchanging messages in- 
teractively and instantly through the Internet, 
until said Master machine posting its current sam- 



pling k > k to said web server, then, 

m 

said web server sending said response to said 
Learner machine, and 

said Learner machine releasing blocking to con- 
tinue said simulation; 
creating a Master-Machine-Learner interaction loop, 
wherein said Master-Machine-Learner interaction loop is 
a dynamic mechanism that couples said Master with said 
Learners virtually through said software-2 machines over 
the Internet. 

[c67] The method of claim 66, wherein said step of exchang- 
ing messages instantly through the Internet further com- 
prises the steps of: 

providing a machine, wherein said machine has said cur- 
rent sampling k= = k ; 

m 

said machine sending a message to said web server, 
wherein said message is selected from the group con- 
sisting of a text, an animation, a script command, a URL, 
and combination thereof; and 

web server sending said message to all other machines 
that have said current sampling k= = k instantly. 

m 

[c68] The method of claim 65, wherein said step of communi- 
cating among said plurality of software-2 machines in- 
teractively through the Internet in real-time, further 



comprises the steps of: 

providing at least one service over the Internet to said 
group of Learner machines by at least one service 
provider, wherein said service is a multimedia service 
delivered in real-time; 

controlling said service by said Master machine; 
synchronizing said service with said Master machine and 
said Master. 

[c69] The method of claim 65, further comprising the steps of: 
providing a Master machine, wherein said Master ma- 
chine is run interactively by a Master; 
providing a simulation service over the Internet through 
real-time communication, wherein said simulation ser- 
vice is published by said Master machine; 
providing at least one Learner machine, wherein said 
Learner machine is run interactively by a Learner; 
wherein said Learner machine subscribes to said simula- 
tion service over the Internet through real-time commu- 
nication; 

providing a Master-Machine-Learner loop over the Inter- 
net, wherein said Master-Machine-Learner loop is a Hu- 
man-Machine-Human interaction running the same soft- 
ware-2 machines in sync through real-time communica- 
tion. 



[c70] The method of claim 69, further comprising the steps of: 
providing a book, wherein said book is authored by pro- 
gramming said software controller to drive said software; 
projecting at least one of, 

(a) a course trajectory running over said software, 
wherein said course trajectory simulates simulation 
of the underlying domain knowledge, and 

(b) a Lab trajectory running over said software, 
wherein said Lab trajectory simulates operation of 
said software; 

wherein said simulation of the underlying domain 
knowledge is programmed in said software; 
wherein said operation of said software is programmed 
in said software; 

identifying said model of said software in said software 
modeling process; 

structuring said model of said software using chapters 
and keyword indices based on said index component G; 
extending a new subject understanding into said model 
of said software based on said programmable extension 
component E; 

simulating a course content interactively over the Inter- 
net in said Master-Machine-Learner loop or locally, 
wherein said course content is programmed in said 
course trajectory; 

simulating a Lab track interactively over the Internet in 



said Master-Machine-Learner loop or locally, wherein 
said Lab track is programmed in said lab trajectory; 
organizing teaching-learning activities in a web of soft- 
ware-2 simulation activities. 

[c71] A system for modeling and simulating software running 
interactively directly or indirectly on at least one digital 
computer, comprising: 
at least one processor; 
a display; 

a computer readable medium; 

a memory, wherein said memory is a part of said com- 
puter readable medium coupled to said processor; 
an input, wherein said input connects at least one of a 
pointing device, a keyboard and external interactive de- 
vices to said software; 

an output, wherein said output connects a block of 
memory to said display; 

a software controller stored on said computer readable 
medium, wherein said software controller is a pro- 
grammable agent controlling said software to perform 
tasks; 

a software modeling process implemented on said com- 
puter readable medium, wherein said software modeling 
process models an interaction process executed on said 
processor between said software and said software con- 



troller, further configured to, 

(a) connect said software controller with said soft- 
ware through said input and said output of said soft- 
ware, 

(b) control said software by said software controller, 
and 

(c) identify a model of said software on-line; and 
a software simulation process implemented on said 
computer readable medium, wherein said software simu- 
lation process simulates said interaction process exe- 
cuted on said processor between said software and said 
software controller, further configured to: 

(d) connect said software controller with said model 
of said software through a simulated input and a 
simulated output of said model of said software, 

(e) control said model of said software by said soft- 
ware controller, and 

(f) simulate said interaction process between said 
software and said software controller without said 
software presence; and 

wherein said software simulation process is a new soft- 
ware that includes said model of said software and said 
software controller. 

[c72] The system of claim 71, further comprising: 

a discrete sampling domain, wherein said discrete sam- 



pling domain is a finite integer sequence K with a current 
sampling k indicating the most recent sampling. 

[c73] The system of claim 72, further comprising: 

a software dynamic system to represent said software 
modeling process, wherein said software dynamic sys- 
tem is a discrete system defined over said discrete sam- 
pling domain K; 

wherein said software simulation process simulates said 
software dynamic system. 

[c74] The system of claim 73, further comprising: 

an algorithm engine residing in said memory and a cur- 
sor engine residing in said memory , wherein said soft- 
ware is factored into said algorithm engine and said cur- 
sor engine; 

a model of said algorithm engine, wherein said model of 
said algorithm engine is identified in said software mod- 
eling process; 

a model of said cursor engine residing in said memory, 
wherein said model of said cursor engine is identified in 
said software modeling process; 
wherein said model of said algorithm engine and said 
model of said cursor engine are controlled in parallel by 
said software controller in said software simulation pro- 
cess; and 

wherein an output of said model of said algorithm en- 



gine and an output of said model of said cursor engine 
are combined as said simulated output of said model of 
said software in said software simulation process. 

[c75] The system of claim 74, wherein said cursor engine is 
modeled, identified and simulated, further comprising: 
a cursor function residing in said memory, wherein said 
cursor function calculates a cursor output based on a 
cursor position, a data structure and a plurality of cursor 
images; 

wherein said cursor function models said cursor engine 
analytically; 

a cursor engine modeler residing in said memory, 
wherein said cursor engine modeler identifies said data 
structure and said plurality of cursor images while said 
cursor engine is controlled by said software controller in 
said software modeling process; 
wherein said model of said cursor engine under control 
of said software controller comprises said cursor func- 
tion with said identified data structure and said identi- 
fied cursor images; 

wherein said model of said cursor engine simulates said 
cursor engine in said software simulation process, 
wherein said software simulation process is executed on 
said processor, further configured to: 

control said model of said cursor engine by said 



software controller; 

calculate said cursor output by said cursor function 
based on, 
said identified data structure, 
said identified cursor images, and 
said cursor position that is supplied by said soft- 
ware controller; 
whereby said cursor engine is simulated by said model 
of said cursor engine that is controlled by said software 
controller in said software simulation process without 
said software presence. 

[c76] The system of claim 74, wherein said model of said al- 
gorithm engine is modeled and identified in said soft- 
ware modeling process, further comprising: 
an initial state of said algorithm engine residing in said 
memory, wherein said initial state of said algorithm en- 
gine is sampled at the first output of said software; 
a update function residing in said memory, wherein said 
update function is a sequence of discrete updates over 
said discrete sampling domain; 

wherein said update function with said initial state model 
said algorithm engine; 

an algorithm engine modeler residing in said memory, 
wherein said algorithm engine modeler samples said se- 
quence of discrete updates while said software is con- 



trolled by said software controller in said software mod- 
eling process; 

wherein said model of said algorithm engine identified 
comprises said sampled initial state and said sampled 
sequence of discrete updates as a modeled algorithm 
engine. 

[c77] The system of claim 76, wherein said algorithm engine 
modeler is controlled by said software controller and 
said algorithm engine. 

[c78] The system of claim 77, wherein said algorithm engine 
modeler is controlled by two sub-samplers, further com- 
prising: 

a direct input and output sampler residing in said mem- 
ory; 

an internal computation sampler residing in said mem- 
ory; and 

wherein said direct input and output sampler and said 
internal computation sampler run mutually exclusively. 

[c79] The system of claim 78, wherein said direct input and 
output sampler controls said algorithm engine modeler 
to sample at least one discrete update induced directly 
by a command from said software controller, further 
comprising means for: 

issue said command by said software controller; 



wait for T(a), wherein said oc < 1; 
call said direct input and output sampler by said soft- 
ware controller; 

trigger said algorithm engine modeler by said direct in- 
put and output sampler to sample said discrete update 
D(k+tx)-D(k); 

hold said command by said software controller; 
advance said current sampling k by one; 
model D(k+a) as D(k+1). 

[c80] The system of claim 79, wherein said direct input and 
output sampler is callable by said software controller 
programmatically. 

[c81] The system of claim 78, wherein said internal computa- 
tion sampler controls said algorithm engine modeler to 
sample at least one discrete update that is driven by an 
internal computation flow inside said algorithm engine, 
further comprising: 

(a) an internal computation flow residing in said memory, 
wherein said internal computation flow is executed by 
said algorithm engine; 

(b) at least one anchored point in said internal computa- 
tion flow, wherein said anchored point is an internal 
computation unit of said algorithm engine with its com- 
putation state transition to be sampled precisely; 

(c) a software sensor residing in said memory, wherein 



said software sensor is attached into said anchor point; 

(d) means for setting up said software sensor with a 

n u m be rOf States argument, wherein said num be rOf States 
argument specifies computation state transitions to be 
observed; 

(e) means for driving said algorithm engine to said inter- 
nal computation unit; 

(f) means for calling said software sensor before said in- 
ternal computation unit is processed; 

(g) means for processing said internal computation unit; 

(h) means for calling said software sensor after said in- 
ternal computation unit is processed. 

(i) means for subtracting said numberOf States argument 
by one; 

(j) a logic residing in said memory to check said num- 
berOfStates argument is not zero, wherein if said logic is 
true, going back to (e); otherwise 
(k) means for removing said software sensor. 

[c82] The system of claim 81, wherein said software sensor is 
an Event Probe that may perform any programmed func- 
tion including triggering said sampling by said algorithm 
engine modeler, further comprising: 
an anchored event communicating in said memory, 
wherein said anchored event is a window message; 
wherein said anchored event processing is said anchored 



point; 

means for modifying a callback function address of an 
internal window message processing; 
means for driving said algorithm engine to said internal 
window message processing; 

a dispatch unit residing in said memory, wherein said 
dispatch unit checks a current message with said an- 
chored event, if a match is found, then, 

means for calling a BeforeEvent Probe, wherein said 
BeforeEvent Probe is said Event Probe, and 
means for processing said anchored event process- 
ing, 

means for calling said AfterEvent Probe, wherein said 
AfterEvent Probe is said Event Probe, and 
means for returning from said internal window mes- 
sage processing; 

otherwise, means for processing an internal window 

message process. 

[c83] The system of claim 81, wherein said software sensor is 
an API Probe that may perform any programmed function 
including triggering said sampling by said algorithm en- 
gine modeler, further comprising: 
(a) a marker function residing in said memory, wherein 
said marker function is an application programming call 
that is modeled as said anchored point within said inter- 



nal computation flow; 

(b) means for modifying a binary execution image of said 
software to reroute a calling to said marker function to a 
new function, wherein said new function includes at least 
one of said API probes and a call to said marker function; 

(c) means for driving said algorithm engine to said inter- 
nal computation flow; 

(d) means for proceeding to said marker function; 

(e) means for calling a BeforeMarker probe before said 
marker function is processed, wherein said BeforeMarker 
is said API Probe; 

(f) means for processing said marker function; 

(g) means for calling an AfterMarker probe after said 
marker function is processed, wherein said AfterMarker 
is said API Probe; 

(h) means for subtracting said numberOf States argument 
by one; 

(i) a logic residing in said memory to check said num- 
berOfStates argument is not zero, wherein if said logic is 
true, going back to (c); otherwise 

(j) means for controlling said algorithm engine to stop 
said processing of said internal computation flow; 
(k) means for removing said BeforeMarker and After- 
Marker probes. 

[c84] The system of claim 81, further comprising: 



a controlled target residing in said memory, wherein said 
controlled target is said software in said software model- 
ing process, or said model of said software in said soft- 
ware simulation process; 

an input of said controlled target residing in said mem- 
ory, wherein said input of said controlled target is said 
input of said software in said software modeling process, 
or said simulated input of said model of said software in 
said software simulation process; 

an output of said controlled target residing in said mem- 
ory, wherein said output of said controlled target is said 
output of said software in said software modeling pro- 
cess, or said simulated output of said model of said 
software in said software simulation process. 

[c85] The system of claim 84, wherein said software controller 
further comprises, 

a work-flow controller residing in said memory, wherein 
said work-flow controller controls said controlled target 
to manipulate at least one user interface; and 
a work-space controller residing in said memory, 
wherein said work-space controller drives said controlled 
target to perform at least one design function. 

[c86] The system of claim 85, wherein said work-space con- 
troller comprises at least one of a coordinate controller 
residing in said memory and a functional controller re- 



siding in said memory. 

[c87] The system of claim 86, wherein said coordinate con- 
troller performs said design function, further compris- 
ing: 

a list of coordinates residing in said memory, wherein 
said list of coordinates are used to drive said input of 
said controlled target. 

[c88] The system of claim 86, wherein said functional con- 
troller performs said design function, further compris- 
ing: 

at least one modelable function residing in said memory, 
wherein said modelable function is called with parame- 
ters to calculate coordinates; 

wherein said coordinates are used to drive said input of 
said controlled target. 

[c89] The system of claim 88, further comprising: 

at least one software actuator residing in said memory, 
wherein said software actuator connects said software 
controller to said controlled target. 

[c90] The system of claim 89, wherein said software actuator 
models at least one graphical user interface element of 
said controlled target. 

[c91] The system of claim 90, wherein said software actuator 



comprises a work-flow actuator interfacing to said work- 
flow controller of said software controller, further com- 
prising means for: 

sending a high level command from said work-flow con- 
troller to said work-flow actuator, wherein said high 
level command comprises a tag and an input command; 
calculating a valid region based on said tag by said 
work-flow actuator; 

translating said command to at least one primitive action 

by said work-flow actuator; 

if said command is for said pointing device, then 

computing a target position by said work-flow actu- 
ator based on said valid region, 
driving said input of said controlled target to said 
target position by said work-flow actuator, and 
applying said primitive action to said target by said 
work-flow actuator; 

else, 

applying said primitive action to said valid region by 
said work-flow actuator. 

[c92] The system of claim 91, wherein said means for calculat- 
ing said valid region based on said tag further comprises 
means for: 

if running in said software modeling process, then 
(a) powering said valid region by said software, 



(b) identifying said valid region based on said tag 
on-line, 

(c) calculating said valid region, and 

(d) saving said valid region with said tag as a part of 
said model of said software; 

otherwise, running in said software simulation process, 

(e) simulating said valid region by said model of said 
software, 

(f) retrieving said valid region based on said tag from 
said model of said software, and 

(g) calculating said valid region. 

[c93] The system of claim 92, wherein said means for identify- 
ing said valid region based on said tag in said software 
modeling process further comprises: 
a unique textual name residing in said memory, wherein 
said unique textual name is said tag; 
a unique relation residing in said memory, wherein said 
unique relation is coded in a string and assigned as said 
unique textual name; 

means for finding a unique handle from said software 
based on said tag, wherein said handle is uniquely lo- 
cated by said unique relation that is coded in said tag; 
means for identifying said valid region based on said 
unique handle that is powered by said software. 



[c94] The system of claim 93, wherein said unique relation is a 
parent-children-sibling relation that is defined uniquely 
by said graphical user interface element of said software. 

[c95] The system of claim 90, wherein said software actuator 
comprises a work-space actuator interfacing to said 
work-space controller of said software controller, further 
comprising means for: 

(a) sending a high level command from said work-space 
controller to said work-space actuator; 

(b) calculating a coordinate by said work-space actuator; 

(c) translating said high level command to at least one 
primitive action by said work-space actuator; 

(d) driving said input of said controlled target to said co- 
ordinate by said work-space actuator; 

(e) applying said primitive action to said coordinate by 
said work-space actuator; 

(f) going back to (b) if at least one coordinate remains. 

[c96] The system of claim 95, wherein said coordinates are 

calculated by said work-space actuator, further compris- 
ing: 

a list of coordinates residing in said memory, wherein 
said list of coordinates is an array of elements with each 
element having a coordinate (x,y); 
wherein said coordinate is retrieved sequentially from 
said array until the end of said array is reached. 



[c97] The system of claim 95, wherein said coordinates are 

calculated by said work-space actuator, further compris- 
ing: 

at least one modelable function residing in said memory, 
wherein said modelable function calculates said coordi- 
nate based on at least one parameter that is supplied by 
said software controller; 

wherein said modelable function implemented by said 
work-space actuator is a mathematical function that is 
scalable. 

[c98] The system of claim 90, wherein said software actuator 
is a system actuator if a system standard graphical user 
interface element is modeled by said software actuator. 

[c99] The system of claim 90, wherein said software actuator 
is a custom actuator if a custom-built graphical user in- 
terface element is modeled by said software actuator. 

[d00] The system of claim 98, wherein said system actuator is 
reusable universally for a plurality of said software mod- 
eling processes and said software simulation processes. 

[c101] The system of claim 99, wherein said custom actuator is 
reusable for a plurality of said software controllers for 
said software modeling process and said software simu- 
lation process. 



[d02] The system of claim 92, wherein said software actuator 
installs at least one software sensor. 

[d03] The system of claim 92, further comprising: 

a running context of said controlled target residing in 
said memory, wherein said running context is dynami- 
cally reconstructed by said software actuator in said 
software simulation process; 

a structured input and output residing in said memory, 
wherein said structured input and output structuralizes 
said input and output of said controlled target based on 
said running context; 

a feedback residing in said memory from said controlled 
target to said software controller through said structured 
output of said controlled target; 

means for inferring a causal relationship from said run- 
ning context. 

[d04] The system of claim 89, further comprising means for: 
modularizing said software modeling process and said 
software simulation process; 

replacing said software with said model of said software 
that is identified in said software modeling process; 
switching from said software modeling process to said 
software simulation process; 

preserving said connection between said software con- 



troller and said software actuator so that said software 
controller is kept invariant in said modeling process and 
said simulation process. 

[d05] The system of claim 89 wherein said modeled algorithm 
engine is controlled by said software controller in said 
software simulation process further comprises: 
a Dynamic Update residing in said memory, wherein said 
Dynamic Update simulates said output of said algorithm 
engine; 

means for reconstructing an output of said modeled al- 
gorithm engine, wherein said output of said modeled al- 
gorithm engine is updated by said Dynamic Update in 
sync with said software controller; 
at least one simulated sampler residing in said memory, 
wherein said simulated sampler simulates said sampler 
used in said software modeling process; 
means for controlling said Dynamic Update to update 
one output of said modeled algorithm engine by said 
simulated sampler if said sampler samples one discrete 
update of said algorithm engine in said software model- 
ing process. 

[d06] The system of claim 105 wherein said simulated sampler 
is controlled by said software controller through said 
software actuator. 



[d07] The system of claim 106 wherein said simulated sampler 
is a simulated direct input and output sampler that sym- 
metrically simulates said direct input and output sampler 
used in said software modeling process. 

[d08] The system of claim 106 wherein said simulated sampler 
is a simulated internal computation sampler that sym- 
metrically simulates said internal computation sampler 
used in said software modeling process. 

[d09] The system of claim 105 wherein said Dynamic Update 
downloads and streams from an external source to up- 
date said output of said modeled algorithm engine. 

[c110] The system of claim 109 wherein said external source is 
at least one local file. 

[cm] The system of claim 109 wherein said external source is 
at least one Internet server. 

[c112] The system of claim 107 wherein said software actuator 
drives said simulated direct input and output sampler, 
further comprising means for: 
modeling a sampling that is induced directly by said 
command from said software controller; 
simulating said sampling by calling said simulated direct 
input and output sampler. 



[c113] The system of claim 107 wherein said software actuator 
simulates said internal computation flow of said algo- 
rithm engine, further comprising means for: 
modeling an sampling event, wherein said sampling 
event samples after or before said internal computation 
unit is processed; 

simulating said sampling event by calling said simulated 
internal computation sampler. 

[d 14] The system of claim 106 further comprising: 

a simulated discrete sampling domain residing in said 
memory, wherein said simulated discrete sampling do- 
main is created from the execution flow of said software 
controller in said software simulation process; 
wherein said simulated discrete sampling domain is 
identical causally to said discrete sampling domain. 

[c115] The system of claim 114 further comprising: 

a compressing transformation residing in said memory 
on said simulated discrete sampling domain, wherein 
said compressing transformation reduces a time dura- 
tion between two samplings in said software simulation 
process while causality is preserved; 
a stretching transformation residing in said memory on 
said simulated discrete sampling domain, wherein said 
stretching transformation extends a time duration be- 
tween two samplings in said software simulation process 



while causality is preserved. 

[c116] The system of claim 73, wherein said software modeling 
process is a software modeling automation that runs au- 
tonomously. 

[c117] The system of claim 116 wherein said software simula- 
tion process is a software simulation automation that 
runs autonomously, further comprising: 
an output of said software simulation automation resid- 
ing in said memory, wherein said output of said software 
simulation automation is said output of said model of 
said software that is manipulable. 

[c118] The system of claim 117 wherein said software simula- 
tion automation is augmented with additional computa- 
tion while said software dynamic system is preserved. 

[c119] The system of claim 118 wherein said software simula- 
tion automation is augmented, further comprising: 
an interaction input component H residing in said mem- 
ory, wherein said interaction input component H engages 
a user to interact with said software simulation automa- 
tion; 

an index component G residing in said memory, wherein 
said index component G controls visibility of said output 
of said software simulation automation; 



a programmable extension component E residing in said 
memory, wherein said programmable extension compo- 
nent E extends programmatically said software simula- 
tion automation with additional computational process. 

[d20] The system of claim 119 wherein said interaction input 
component H is inserted between said output of said 
software controller and said input of said model of said 
software, further comprising means for: 

(a) updating said software simulation automation at a 
current sampling k; 

(b) executing a programmed interaction by said software 
controller; 

(c) updating a running context, wherein said running 
context includes a valid region, a pointing device status, 
a keyboard status, and a current command which are 
generated from said programmed interaction by said 
software controller; 

(d) transferring execution of said software simulation 
automation to said H; 

(e) waiting for an interaction input from said user; 

(f) receiving said interaction input from said user; 

(g) comparing said interaction input with said running 
context; 

(h) if said interaction input is matched with said running 
context, then continuing said execution of said software 



simulation automation; 

(i) otherwise, going back to (e). 

[c121] The system of claim 119 wherein said interaction input 
component H is independent of said software simulation 
automation and said H is applicable universally to any 
software simulation automation. 

[d22] The system of claim 119 wherein said interaction input 
component H can be plugged or unplugged into said 
software simulation automation programmatically so that 
said software simulation automation can run interactively 
or automatically. 

[c123] The system of claim 119 wherein said index component 
G indexes said software simulation automation selec- 
tively, further comprising: 

(a) at least one index set K residing in said memory, 
wherein said K is a subset of said discrete sampling do- 
main K; 

(b) means for updating said output of said software sim- 
ulation automation in said memory that is invisible; 

(c) a gated output-mapping function residing in said 

memory, wherein said gated output-mapping function 

controls copying from said memory to said display based 

on said index set K ; 

i 

(d) means for running said software simulation automa- 



tion; 

(e) means for updating a current output of said software 
simulation automation at a current sampling k in said 
memory; 

(f) if said current sampling k g K , then, 

means for copying said current output from said 
memory to said display by said gated output-map- 
ping function, 

means for continuing said simulation; 

(g) otherwise, 

means for advancing said current sampling k by one, 
means for going back to (e) as fast as computation- 
ally possible. 

[d24] The system of claim 123 further comprising: 

first textual term residing in said memory, wherein said 
first textual term is associated with a sampling k ; 
second textual term residing in said memory, wherein 
said second textual term is associated with a sampling k 

2 ; 

at least one pair of descriptive textual terms residing in 
said memory for searching, wherein one in said descrip- 
tive textual terms is said first textual term, and said 
other in said descriptive textual terms is said second 
textual term; 

means for conducting searches based on said descriptive 



textual terms to find two samplings k and k^, 

means for running said software simulation automation; 

means for updating said current output of said software 

simulation automation in said memory; 

if said current sampling k >= kl and k < k2, then 

means for copying said output of said software sim- 
ulation automation from said memory to said display 
by said gated output-mapping function; 
means for selectively running at least one segment of 
simulation from k to visually while computationally 
running said software simulation automation internally. 

[d25] The system of claim 119 wherein said programmable ex- 
tension component E extends additional computation 
into said output of said software simulation automation, 
further comprising: 

a new programmable function, wherein said new pro- 
grammable function implements a sub-process; 
means for updating a current output of said software 
simulation automation at said current sampling k; 
means for extending said new sub-process with said 
current output as an initial state and a next output of 
said software simulation automation as a final state; 
means for computing at least one sub-state with said 
new programmable function, wherein said sub-state in 
said sub-process updates over said output of said soft- 



ware simulation automation; 

means for updating said next output of said software 
simulation automation at said current sampling k+1. 

[d26] The system of claim 125 wherein said new pro- 
grammable function is a Microsoft Internet Explorer 
Browser component, further comprising: 
a DHTML page residing in said memory, wherein said 
DHTML page is loaded by said Microsoft Internet Ex- 
plorer Browser component; 

a Document Object Model (DOM) residing in said mem- 
ory, wherein said Document Object Model is pro- 
grammed in said DHTML page; 

a URL link residing in said memory, wherein said URL link 
is encoded in said DHTML page; 
a target region residing in said memory, wherein said 
target region is defined by said Document Object Model; 
means for clicking or typing in said target region; 
means for navigating to said URL link. 

[d27] The system of claim 120 wherein said software simula- 
tion automation further comprises: 
a controlled target residing in said memory, wherein said 
controlled target is a Microsoft Internet Explorer Browser 
component; 

means for lively running said Internet Explorer Browser 
component in said software simulation automation; 



means for programming web browsing activities by said 
software controller; 

means for engaging said user with said web browsing 
activities interactively or automatically. 

[d28] The system of claim 119 further comprising: 

a software amplification device residing in said memory, 
wherein said software amplification device comprises 

said software controller, 

said interaction input component H, 

said index component G, and 

said programmable extension component E; 
wherein said software amplification device connects with 
said model of said software in a top loop; 
an augmented output of said model of said software re- 
siding in said memory, wherein said augmented output 
of said model of said software is fed through said soft- 
ware amplification device; 

a user, wherein said user senses said augmented output 
as an input from said software amplification device; 
wherein said user is engaged with said software amplifi- 
cation device in a bottom loop; 

an output residing in said memory to said software am- 
plification device, wherein said output to said software 
amplification device is an interactive movement by said 
user; and 



means for interacting said model of said software by said 
user through said software amplification device. 

[d29] The system of claim 128 further comprising means for: 

(a) running a sub-process in said programmable exten- 
sion component E to prepare said user to take a right in- 
teraction, wherein said sub-process implements at least 
one programmable function; 

(b) executing a programmed interaction, wherein said 
programmed interaction is performed by said software 
controller; 

(c) updating a running context, wherein said running 
context includes a valid region, a pointing device status, 
a keyboard status, and a current command which are 
generated from said programmed interaction by said 
software controller; 

(d) waiting in said interaction input component H until 
said user taking a right interaction input against said 
running context; 

(e) scoring said interaction input by said H; 

(f) if said score is higher than a level assigned to a next 
output of said software simulation automation, then un- 
der control of said index component G, 

updating said next output of said software simula- 
tion automation invisibly, 
forgoing a next sub-process, 



canceling a next programmed interaction; 

(g) otherwise, visually updating said next output of said 
software simulation automation; 

(h) going back to step (a); 

(i) or, reaching a final of said software simulation au- 
tomation. 

[d30] The system of claim 128 wherein said interaction input 
component H, said index component G, and said pro- 
grammable extension component E are implemented in 
modular ways so that each component can be bypassed 
programmatically. 

[c131] The system of claim 119 further comprising: 

a software intelligence device residing in said memory, 
wherein said software intelligence device comprises 

said model of said software, 

said interaction input component H, 

said index component G, and 

said programmable extension component E; 
wherein said software intelligence device connects with 
said software controller; 

wherein said model of said software further comprises 
a setup part residing in said memory, 
a modeled algorithm engine part residing in said 
memory, and 

an output part residing in said memory; 



wherein said setup part is simulated invisibly under con- 
trol of said index component G; 

wherein said modeled algorithm engine part is extended 
with an input reasoning by said programmable extension 
component E; 

wherein said output part is extended with an output rea- 
soning by said programmable extension component E; 
means for interactively simulating modeled algorithm 
engine through said interaction input component H by a 
human user. 

[c132] The system of claim 119 further comprising: 

a software-2, wherein said software-2 comprises an au- 
tomation loop and an interaction loop; 
wherein said automation loop simulates and augments a 
modeled software dynamic system that runs au- 
tonomously; 

wherein said interaction loop engages a human user in- 
teractively into said automation loop; 
wherein said interaction loop couples with said automa- 
tion loop programmatically. 

[d33] The system of claim 132 further comprising: 
at least one web server; 

a plurality of same software-2 residing in said memory, 
wherein said plurality of same software-2 are connected 
to said web server through the Internet; 



wherein each of said plurality of same software-2 is run 
interactively by a user locally; 

a Master machine, wherein said Master machine is se- 
lected from said plurality of software-2; 
a Master, wherein said Master runs said Master Machine; 
a group of Learner machines, wherein said group of 
Learner machines are the rest of said plurality of soft- 
ware-2; 

a group of Learner, wherein each Learner inside said 
group of Learners runs each Learner machine inside said 
group of Learner machines; 

a HTTP request/response protocol, wherein said HTTP 
request/response protocol is used to control the execu- 
tion of said group of Learner machines by said Master 
machine; 

means for synchronizing said state of said Learner ma- 
chines with said state of said Master machine interac- 
tively by said Master through said web server; 
means for communicating among said plurality of soft- 
ware-2 machines interactively through the Internet in 
real-time. 

[d34] The system of claim 133 further comprising means for, 
communicating between said Master machine and said 
web server, further comprising means for: 

(a) transiting by said Master machine to a state, 



wherein said state expects an interaction input from 
said Master, 

(b) said Master machine posting a request including a 
current sampling k to said web server, 

m 

(c) said Master machine waiting for a response from 
said web server; 

(d) said web server recording said request informa- 
tion including said k , 

m 

(e) said web server sending said response to said 
Master, and 

(f) releasing said Master machine to continue said 
simulation, 

(g) said Master machine pausing said simulation for a 
local interaction input from said Master in said local 
interaction input component H, 

(h) said Master machine exchanging message with 
said group of Learner instantly through the Internet 
in real-time while said local interaction input compo- 
nent H waits for said interaction input, 

(i) said Master making said interaction input to said 
simulation, and 

(j) said Master machine continuing said simulation; 
and 

communicating between any one of said Learner ma- 
chines and said web server, further comprising means 
for: 



(k) said Learner machine transiting to a state, 
wherein said state expects an interaction input from 
said Learner, 

(I) said Learner machine posting a request including a 
current sampling k to said web server, 
(m) said Learner machine waiting for a response from 
said web server, 

(n) said web server comparing between said k and 

said k of said Master machine recorded most re- 
in 

cently, 

if k < k then, 

m 

said web server sending said response to said 

Learner machine immediately, 

said Learner machine continuing said simulation, 

and 

if k >= k , then, 

m 

said web server suspending said response to said 
Learner machine, and 

said Learner machine blocking said simulation, 
if k == k , then, 

m 

said Learner machine exchanging messages in- 
teractively and instantly through the Internet, 
until said Master machine posting its current sam- 
pling k > k to said web server, then, 



said web server sending said response to said 
Learner machine, and 

said Learner machine releasing blocking to con- 
tinue said simulation; 
creating a Master-Machine-Learner interaction loop, 
wherein said Master-Machine-Learner interactive loop is 
a dynamic mechanism that couples said Master with said 
Learners virtually through said software-2 machines over 
the Internet. 

[c135] The system of claim 134 wherein said exchanging mes- 
sages instantly through the Internet further comprises: 
a machine, wherein said machine has said current sam- 
pling k= = k ; 

m 

means for said machine sending a message to said web 
server, wherein said message is selected from the group 
consisting of a text, an animation, a script command, a 
URL, and combination thereof; and 
means for said web server sending said message to all 
other machines that have said current sampling k= = k 

m 

instantly. 

[d36] The system of claim 133 wherein said communicating 
among said plurality of software-2 machines interac- 
tively through the Internet in real-time, further com- 
prises: 



at least one service over the Internet to said group of 
Learner machines by at least one service provider, 
wherein said service is a multimedia service delivered in 
real-time; 

means for controlling said service by said Master ma- 
chine; 

means for synchronizing said service with said Master 
machine and said Master. 

[c137] The system of claim 133 further comprising: 

a Master machine, wherein said Master machine is run 
interactively by a Master; 

a simulation service over the Internet through real-time 
communication, wherein said simulation service is pub- 
lished by said Master machine; 

at least one Learner machine , wherein said Learner ma- 
chine is run interactively by a Learner; 
wherein said Learner machine subscribes to said simula- 
tion service over the Internet through real-time commu- 
nication; 

a Master-Machine-Learner loop over the Internet, 
wherein said Master-Machine-Learner loop is a Human- 
Machine-Human interaction running the same software- 
2 machines in sync through real-time communication; 

[d38] The system of claim 137 further comprising: 

a book residing in said memory, wherein said book is 



authored by programming said software controller to 
drive said software; 
projecting at least one of, 

(a) a course trajectory residing in said memory run- 
ning over said software, wherein said course trajec- 
tory simulates simulation of the underlying domain 
knowledge, and 

(b) a Lab trajectory residing in said memory running 
over said software, wherein said Lab trajectory simu- 
lates operation of said software; 

wherein said simulation of the underlying domain 
knowledge is programmed in said software; 
wherein said operation of said software is programmed 
in said software; 

means for identifying said model of said software in said 
software modeling process; 

means for structuring said model of said software using 
chapters and keyword indices based on said index com- 
ponent G; 

means for extending a new subject understanding into 
said model of said software based on said programmable 
extension component E; 

means for simulating a course content interactively over 
the Internet in said Master-Machine-Learner loop or lo- 
cally, wherein said course content is programmed in said 
course trajectory; 



means for simulating a Lab track interactively over the 
Internet in said Master-Machine-Learner loop or locally, 
wherein said Lab track is programmed in said lab trajec- 
tory; 

means for organizing teaching-learning activities in a 
web of software-2 simulation activities. 

[d39] A method for modeling and simulating software running 
interactively directly or indirectly on at least one digital 
computer, comprising the steps of: 
providing a first software, wherein said first software is a 
binary software that is runnable in the form of EXE or 
Dynamic Link Libraries (DLL); 

providing a second software, wherein said second soft- 
ware is programmed to synthesize at least one of a plu- 
rality input actions, wherein said plurality of input ac- 
tions comprise, 

at least one pointing device action, 

at least one keyboard action, and 

at least one external input action; 
executing said first software under control of said sec- 
ond software by applying said synthesized input actions 
to said first software by said second software; 
capturing a model of said first software, wherein said 
model of said first software records an input and output 
behavior of said first software under control of said sec- 



ond software; 

controlling said model of said first software by said sec- 
ond software to simulate said input and output behavior 
of said first software under control of said second soft- 
ware; 

creating a third software, wherein said third software 
comprises said model of said first software and said sec- 
ond software. 

[d40] A system for modeling and simulating software running 
interactively directly or indirectly on at least one digital 
computer, comprising: 
at least one processor; 
a computer readable media; 

a memory, wherein said memory is a part of said com- 
puter readable medium coupled to said processor; 
a first software residing in said memory, wherein said 
first software is a binary software that is runnable in the 
form of EXE or Dynamic Link Libraries (DLL); 
a second software residing in said memory, wherein said 
second software is programmed to synthesize at least 
one of a plurality input actions, wherein said plurality of 
input actions comprise; 

at least one pointing device action, 

at least one keyboard action, and, 

at least one external input action; 



wherein said processor is configured to, 
execute said first software under control of said second 
software by applying said synthesized input actions to 
said first software by said second software, 
capture a model of said first software, wherein said 
model of said first software records an input and output 
behavior of said first software under control of said sec- 
ond software; 

control said model of said first software by said second 
software to simulate said input and output behavior of 
said first software under control of said second software; 
a third software residing in said computer readable me- 
dia, wherein said third software comprises said model of 
said first software and said second software. 



